Development of a Computer-Aided Language Learning System for Mandarin – Tone Recognition and Pronunciation Error Detection

This paper reports on the continued activities towards the development of a computer-aided language learning system for teaching Mandarin to Germans. A method for f0 normalization based on maximum likelihood estimation and tone recognition was implemented. Furthermore, a method for detecting the pronunciation errors was tested by calculating the confidence distance between the first and second candidates of the recognition system. In the first experiments we used an Automatic Speech Recognition (ASR) system with an acoustic model trained on data of native speakers of Mandarin. The performance of the ASR system was too poor because it was not adapted to the errors expected from the German learners of Mandarin. In the current experiment we modified the ASR system by considering the most frequent pronunciation errors committed by the German learners using a well-targeted replacement list for every phoneme and adaptation of the acoustic model using the correct data from German learners of Mandarin. The modified ASR system performs better than the original one, but stills falls short of the performance of the human judges.